{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a href='http://www.holoviews.org'><img src=\"../../assets/hv+bk.png\" alt=\"HV+BK logos\" width=\"40%;\" align=\"left\"/></a>\n",
    "<div style=\"float:right;\"><h2>Exercise 3: Networks and GeoViews</h2></div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "import holoviews as hv\n",
    "import geoviews as gv\n",
    "import networkx as nx\n",
    "\n",
    "from bokeh.sampledata.airport_routes import airports, routes\n",
    "from holoviews.operation.datashader import datashade, directly_connect_edges, bundle_graph\n",
    "\n",
    "hv.extension('bokeh')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that if you see an error about ``airports.csv`` not existing when running the above line, make sure to run the *Downloading sample data* section of the [Setup notebook](../00_Setup.ipynb).\n",
    "\n",
    "\n",
    "### Example 1\n",
    "\n",
    "In this exercise we will plot the locations of all US airports on a map. Before we get started let's inspect the ``airports`` dataframe using the ``head`` method."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now declare a ``gv.Points`` element from the ``airports`` dataframe plotting the 'Longitude' and 'Latitude', assign the object to a variable, and display it. Be sure to think carefully about what kind of dimensions (key or value) those columns represent.\n",
    "\n",
    "<b><a href=\"#hint1\" data-toggle=\"collapse\">Hint</a></b>\n",
    "\n",
    "<div id=\"hint1\" class=\"collapse\">\n",
    "``Points``, unlike ``Scatter``, has two key dimensions (``kdims``). It represents a two-dimensional space (in this case the position of each airport on a map).\n",
    "</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<b><a href=\"#solution1\" data-toggle=\"collapse\">Solution</a></b>\n",
    "\n",
    "<div id=\"solution1\" class=\"collapse\">\n",
    "<br>\n",
    "<code>points = gv.Points(airports, ['Longitude', 'Latitude'])\n",
    "points</code>\n",
    "</div>\n",
    "\n",
    "You should now be able to see that the dataset includes various US airbases in Europe, in addition to those actually located in US states. Let's focus only on airports in US states by using the ``select`` method to select airports between -180 and 0 in Longitude and above 0 degrees in Latitude, and assign that to your variable.\n",
    "\n",
    "<b><a href=\"#hint2\" data-toggle=\"collapse\">Hint</a></b>\n",
    "\n",
    "<div id=\"hint2\" class=\"collapse\">\n",
    "The select method allows selecting coordinates, multiple values and ranges of values, e.g. to select rows where the 'value' column has values between 0 and 100 use ``dataset.select(column=(0, 100))``.  ``None`` can be used for any range start or end that you do not need to enforce.\n",
    "</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<b><a href=\"#solution2\" data-toggle=\"collapse\">Solution</a></b>\n",
    "\n",
    "<div id=\"solution2\" class=\"collapse\">\n",
    "<br>\n",
    "<code>points = points.select(Longitude=(-180, 0), Latitude=(0, None))\n",
    "points</code>\n",
    "</div>\n",
    "\n",
    "Finally, overlay the points on a map tile source using the ``gv.WMTS`` element and the tile source URL provided below. Then adjust the width and height of the ``Points`` and enable the 'hover' tool.\n",
    "\n",
    "<b><a href=\"#hint3\" data-toggle=\"collapse\">Hint</a></b>\n",
    "\n",
    "<div id=\"hint3\" class=\"collapse\">\n",
    "Tools can be enabled on bokeh plots by supplying a list of tools, e.g. ``tools=['box_select', 'hover']``.\n",
    "</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "url = 'https://maps.wikimedia.org/osm-intl/{Z}/{X}/{Y}.png'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<b><a href=\"#solution3\" data-toggle=\"collapse\">Solution</a></b>\n",
    "\n",
    "<div id=\"solution3\" class=\"collapse\">\n",
    "<br>\n",
    "<code>%%opts Points [width=500 height=400 tools=['hover']] (size=2 color='black' fill_alpha=0)\n",
    "gv.WMTS(url) * points</code>\n",
    "</div>\n",
    "\n",
    "### Example 2\n",
    "In this exercise we will plot a network graph of all the airport connections. As always, first inspect the relevant datasets. We've already seen the structure of the ``airports`` dataset, so let's look at the ``routes`` dataframe as well:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's make a NetworkX graph to work with this data about airport routes:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "g = nx.from_pandas_edgelist(routes, 'SourceID', 'DestinationID')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "``g`` is an abstract graph object, but we can make a visualizable ``hv.Graph`` object from it using the ``hv.Graph.from_networkx`` classmethod. ``Graph.from_networkx`` accepts the NetworkX graph as the first argument and a layout function such as ``nx.spring_layout`` as the second argument. Once it displays, reduce the ``node_size`` so you can make out the different nodes.\n",
    "\n",
    "<b><a href=\"#hint4\" data-toggle=\"collapse\">Hint</a></b>\n",
    "\n",
    "<div id=\"hint4\" class=\"collapse\">\n",
    "``Graph`` style options are split into ``node_`` and ``edge_`` options.\n",
    "</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<b><a href=\"#solution4\" data-toggle=\"collapse\">Solution</a></b>\n",
    "\n",
    "<div id=\"solution4\" class=\"collapse\">\n",
    "<br>\n",
    "<code>%%opts Graph (node_size=4)\n",
    "hv.Graph.from_networkx(g, nx.spring_layout)</code>\n",
    "</div>\n",
    "\n",
    "If you hover over the nodes you will notice that while it includes an index the hover information is otherwise not very useful. We will now add additional node information by supplying a Dataset indexed by the AirportID. \n",
    "\n",
    "First, declare a ``hv.Dataset`` for the ``airports`` with the 'AirportID' as a key dimension and the 'Name' and 'TZ' (or timezone) as value dimensions. Then supply the Dataset as the third argument to the ``Graph.from_networkx`` function and check that the hover now shows more useful information.\n",
    "\n",
    "Finally, customize the plot by adjusting ``width`` and ``height``, adjusting the ``node_size`` and ``edge_line_width`` and add a ``color_index`` and a ``cmap`` of 'tab20'.\n",
    "\n",
    "<b><a href=\"#hint5\" data-toggle=\"collapse\">Hint</a></b>\n",
    "\n",
    "<div id=\"hint5\" class=\"collapse\">\n",
    "Ensure the ``Dataset`` with additional node info defines 'AirportID' as the sole key dimension. \n",
    "</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<b><a href=\"#solution5\" data-toggle=\"collapse\">Solution</a></b>\n",
    "\n",
    "<div id=\"solution5\" class=\"collapse\">\n",
    "<br>\n",
    "<code>%%opts Graph [width=600 height=600 color_index='TZ'] (node_size=4 cmap='tab20' edge_line_width=1)\n",
    "ds = hv.Dataset(airports, 'AirportID', ['Name', 'TZ'])\n",
    "hv.Graph.from_networkx(g, nx.spring_layout, ds)</code>\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Example 3\n",
    "\n",
    "In this exercise we will combine what we learned about geographic data with graph support to visualize the flight connections on a map. Since there are many more connections than we can easily view, we will count the number of connections between airports and select the fifty busiest."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Count the number of connections from each airport\n",
    "counts = routes.groupby('SourceID')[['Stops']].count().reset_index().rename(columns={'Stops': 'Connections'})\n",
    "airports_df = pd.merge(airports, counts, left_on='AirportID', right_on='SourceID', how='left')\n",
    "\n",
    "# Select only airports located in US states & convert from Web Mercator to Latitudes/Longitudes\n",
    "airport_points = gv.Points(airports_df, ['Longitude', 'Latitude']).select(Longitude=(-180, 0), Latitude=(0, None))\n",
    "projected_points = gv.operation.project_points(airport_points)\n",
    "\n",
    "busiest_ids = list(routes.groupby('SourceID').count().sort_values('Stops').iloc[-50:].index.values)\n",
    "tiles = gv.WMTS('https://maps.wikimedia.org/osm-intl/{Z}/{X}/{Y}@2x.png')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We now have the ``AirportIDs`` for the 50 busiest airports defined on the ``busiest_ids`` variable, a tile source defined on the ``tiles`` variable and below we have already defined a number of options and a ``Nodes`` element.\n",
    "\n",
    "Define an hv.Graph from the ``routes`` and ``nodes``, declaring 'SourceID' and 'DestinationID' as the key dimensions. Then use the ``graph.select`` method to select by ``AirportID`` using the ``busiest_ids``. Then overlay the selected Graph on top of the ``tiles``. Also try switching the ``selection_mode`` on the ``select`` method from 'edges' to 'nodes' and observe the difference.\n",
    "\n",
    "<b><a href=\"#hint6\" data-toggle=\"collapse\">Hints</a></b>\n",
    "\n",
    "<div id=\"hint6\" class=\"collapse\">\n",
    "When constructing a ``Graph`` supply the edges and nodes as tuple, e.g. ``hv.Graph((edges, nodes))``.\n",
    "    \n",
    "The ``selection_mode`` is a special keyword of the Graph.select method.\n",
    "</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "%%opts Graph [width=800 height=800] (edge_selection_line_color='black' edge_hover_line_color='red')\n",
    "%%opts Graph (node_size=8 edge_line_width=1 edge_line_alpha=0 edge_nonselection_line_alpha=0)\n",
    "nodes = hv.Nodes(projected_points, ['Longitude', 'Latitude', 'AirportID'], ['Name', 'City', 'Connections'])\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<b><a href=\"#solution6\" data-toggle=\"collapse\">Solution</a></b>\n",
    "\n",
    "<div id=\"solution6\" class=\"collapse\">\n",
    "<br>\n",
    "<code>%%opts Graph [width=800 height=800] (edge_selection_line_color='black' edge_hover_line_color='red')\n",
    "%%opts Graph (node_size=8 edge_line_width=1 edge_line_alpha=0 edge_nonselection_line_alpha=0)\n",
    "nodes = hv.Nodes(projected_points, ['Longitude', 'Latitude', 'AirportID'], ['Name', 'City', 'Connections'])\n",
    "\n",
    "# Declare nodes, graph and tiles\n",
    "graph = hv.Graph((routes, nodes), ['SourceID', 'DestinationID'])\n",
    "\n",
    "# Select 50 busiest airports\n",
    "busiest_airports = graph.select(AirportID=busiest_ids, selection_mode='edges')\n",
    "\n",
    "tiles * busiest_airports</code>\n",
    "</div>"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
